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ABSTRACT 



The evaluations made by 36 Na\^ fighter pilots of 64 air-to-air 
combat situations are statistically analyzed to detect any significant 
differences between pilots' evaluative techniques in relation to their 
rank, flight hours and combat missions. Predictor equations are computed 
and used in a self-analyzing, self-modifying artificial intelligence 
program modeled on an instructor-flight student interactive situation. 
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r. INTRODUCTION 



From the legendary Baron von Richtofen to the popular Snoopy 
flying his doghouse through the comic strips, the image of the fighter 
pilot is one of an enigma capable of performing complex and dangerous 
tasks in an environment where fractions of a second can spell the dif- 
ference between success and filure, life and death. This fascinating 
picture of a highly trained, assured and swaggering superbeing beckons 
to be studied; to have the mystique lifted. Hence, the impetus of this 
research: the fighter pilot. 

Other than the satisfaction of human curiosity, the study of the 
fighter pilot -in this case the Naval fighter pilot- can ultimately result 
in improved training and selection of future pilots by understanding the 
activities of successful pilots; improved combat techniques by adaptation 
of the successful pilots' techniques; improved surface-to-air and air-to- 
air communications through better understanding; and improved electronic 
and mechanical pilot aids to enhance pilot performance. 

A. OBJECTIVES 

The objectives of this research lay in two major areas. The first 
was to address the questions of how Navy fighter pilots evaluate an air- 
to-air combat threat, i.e. , pilot versus pilot situations. How does the 
pilot reach his decision? Can this decision be somehow quantified for 
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computer applications ? What differences exist among pilots in their 
evaluations of. air-to-air threat? 

The second major area lies in the field of artificial intelligence. 
Classically concerned with games such as chess and checkers with well 
defined rules for determining success and failure [1, 2], artificial intel- 
ligence has seen little application in real life "games." A notable 
exception is Clarkson’s portfolio program which has reached partial 
agreement with a successful bank trust administrator [3]. Heuristics, 
such as the minimax technique whereby a player will always attempt to 
minimize his maximum loss [4], and binary choice selection [5], although 
fine for board games, find little application in games of life. As stated 
by Rigney [6]: 

"For decision tasks in which the outcome is of great consequence 
and the time for decision is short, as in many military, business, 
and medical decisions, the strategy the decision maker uses is 
of special interest." 

From the strategies hopefully revealed in the attainment of the first 
objective, an artificial intelligence application is sought. As brought 
out by Shepard [7], an obvious disparity exists between "the effortless- 
ness and surety of most perceptual decisions and the painful hesitation 
and doubt characteristic of these 'higher level’ decisions" and strategies. 
To overcome this disparity in a "game" of ill defined rules and high stakes 
is the objective of the artificial intelligence application. 
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B. . DEFINITIONS 

At this time some definitions of terms used throughout this research 
should be brought forward. Artificial intelligence as used in this study 
encompasses the construction of "computer programs which exhibit 
behavior that we call 'intelligent behavior' when we observe it in human 
beings [8] . " 

The area of artificial intelligence most apt for this study is 
decision making in. game playing. Hence the game is defined as air- 
to-air combat as experienced in the Vietnam area of operations . 

C. ASSUMPTIONS 

Certain assumptions used throughout the study are; 

(1) Data such as military rank and threat evaluation decisions 
can be placed on a quantifiable continuum. 

(2) Since no criteria of deciding success or failure in the game 
exist short of actual combat results, expertise in the game 
will decide successes or losses. 

(3) Expertise is directly proportional to military rank, flight 
hours and total number of combat missions. 

(4) The data base used is a representative sample of Navy 
fighter pilots. 
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n.. EXPERIMENTAL PROCEDURE-DETERMIISIING DECISION STRATEGY 



In an attempt to isolate and identify the important variables con- 
sidered by a fighter pilot in his evaluation of an air-to-air combat 
situation, a mail-out survey was designed. In the survey the subjects 
were asked to assume the following hypothetical situation: 

(1) The subject is an airborne Target Area Combat Air Patrol 
(TARCAP) over enemy territory in Southeast Asia. 

(2) The subject's aircraft is configured and armed for air-to-air 
combat only; hence, a fighter. 

(3) The subject's primary mission is to intercept and destroy 
enemy air raids threatening friendly airborne strike forces. 

(4) The TARCAP and immediate strike group is the only friendly 
force in the area. Therefore, the subject is free to fire at 
any threat without positive visual identification. 

Eleven different items of additional information were then presented 
to the subject. These were presented in pairs yielding a total of 55 
combinations. The items of information were: 

, (1) Relative position of enemy aircraft from the friendly strike 
group . 

(2) Type of enemy aircraft. 

(3) Number of enemy aircraft. 

(4) Number of friendly aircraft. 

(5) Range from friendly aircraft to enemy aircraft. 

(6) Speed of enemy aircraft. 

(7) Friendly aircraft capabilities. 
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(8) Altitude of enemy forces . 

(9) Anti-aircraft activity in the area. 

(10) Bearing of enemy from friendly forces. 

(11 Closure rate of friendly and enemy forces. 

The subject then had to choose which one of the pieces of infor- 
mation presented in each pair would be of greater value in evaluating 
the tlireat of, and in planning engagement tactics against, enemy air 
units . An additional alternative was given whereby each member of the 
pair could be evaluated as being equal in importance. Appendix A con- 
tains a sample of this questionnaire. Ninety-seven Naval aviators 
responded to this questionnaire of which 68 or 70.1% were pilots and 
29 or 29.9% were Radar Intercept Officers, the second crewman in the 
F-4 Phantom jet. Eleven or 11.3% were F-8 Crusader pilots, the rest 
of the pilots being F-4 pilots. Sixty-three or 65.0% of the aviators had 
combat experience over Korea or Vietnam. Nine or 9.4% had engaged in 
actual air-to-air combat. 

From a scaling of the variables from the pair-comparisons data 
the variables were ranked as follows (in descending order of importance) 

(1) Relative position of enemy aircraft. 

(2) Bearing of enemy. 

(3) Range of enemy. 

(4) Altitude of enemy. 

(5) Friendly aircraft capabilities . 
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(6) Number of enemy. 

(7) Rate of closure . 

(8) Anti-aircraft activity. 

(9) Type of enemy aircraft. 

(10) Speed of enemy. 

(1 1) Number of friendly . 

From the results of the first survey, and with the consideration 
of the type of information available to the pilot in the cockpit, a second 
mail -out survey was designed. The variables considered in this survey 
were limited to six in number. Fuel states and rules of engagement were 
added for realism. The remaining four variables were considered as being 
information a pilot would have at his access from either his cockpit 
instruments or via a surface or airborne controller. The variables were 
limited to two states, hence 25 or 64 possible combinations were con- 
sidered by each subject. The variables and their states were: 

(1) Fuel above bingo; the maximum amount of fuel that can be 
utilized before fuel level drops below that necessary for a 
safe return to home base. 

(a) 1000 pounds. 

(b) 2 500 pounds. 

(2) Rules of engagement; when the subject may take the suspected 
aircraft under fire. 

(a) Eyeball (positive visual identification is required). 

(b) Missiles free (positive visual identification is not 
required) . 



12 



(3) Enemy's relative bearing from the subject. 

(a) 315° 

(b) 135° 

(4) Enemy's range from the subject. 

(a) Eyeball or within 4 nautical miles . 

(b) 2 0 nautical miles. 

(5) Enemy's true heading or direction of flight. 

(a) 045° 

(b) 225° 

(6) Number of enemy aircraft . 

(a) 2 

(b) 6 

The subjects were asked for their rank, total number of flight 
hours and total number of combat missions. The follov/ing background 
scenario was used for each of the 64 situations: 

"You are the flight leader of a section^ of F-4's armed with two 
Sparrows^ and two Sidewinders .3 Assume for this exercise that the 
aircraft's weapon systems are up in every respect. You are providing 



A section normally consists of two F-4's. 
Beam riding guided missile. 

Heat seeking guided missile. 
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TARCAP for a division of A-7’s who have just completed a strike and are 
egressing from the target area. You are feet-dry"^ over North Vietnam 
(20 nm to the coast) . The AA and SAM^ defenses in the immediate area 
are light to moderate . You have limited GCI® facilities operating for 
you and the enemy has excellent ground radar control. 

The enemy aircraft are assessed to be MIG-21 's at 15,000 feet and 
500 kts . You are 10,000 feet and 450 kts. heading for your carrier 
(360° relative) . 

The weather in the area is clear and 15+ visibility. There are 
several flights of attack aircraft still feet-dry, exact position unknown. 

The MiG's have demonstrated an air-to-air missile capability." 

Figure 1 shows the way the tactical situations were presented to the 
subjects, analogous to a radar scope or maneuvering board's display. 

Two major evaluation questions were asked of each subject in con- 
junction with each of the 64 situations considered. These questions 
are shown in Figure 2 . 

The questionnaire was sent to a number of Navy fighter squadrons 
on both the East and West Coasts. The aircraft in service at each of 
the squadrons was the F-4 Phantom. Thirty-six pilots responded. These 
pilots arc described in Appendix B. Table 1 contains a description of 
the pilots by rank, hours and combat missions. The overall average 



Over land vice feet wet or over water. 

^Surface-to-Air Missile. 

0 

Ground Control Intercept. 
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Figure 1 

TACTICAL SITUATION PRESENTATION 
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Figure 2 



QUESTIONS ASKED WITH EACH TACTICAL DISPLAY 

A. In this tactical situation (check one answer): 

1.. I'd have no choice; there would be an engagement. 

2 .. I'd have a choice on whether to engage or not, and I'd 

engage . 

3 . I wouldn't engage. 

B. . Indicate what aircraft losses you would predict, if there were an 

engagement . 

Enemy Losses (no. of aircraft) 

Friendly Losses (no. of aircraft) 
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respondee had a rank between Lieutenant and Lieutenant Commander 

7 

(3.47) , 1994.03 flight hours and 124.11 combat missions. Rank was 
assumed to be an attribute providing ordinal data. 

The data from the surveys were coded and placed on a magnetic 
disk for statistical analysis. The rules of engagement, being qualitative 
variables, were arbitrarily coded as a 0 for eyeball and 1 for missiles- 
free. A measure of central tendency (arithmetic mean) and measure of 
dispersion (standard deviation) was computed for each biographical 
variable (rank, flight hours, combat missions), each primary tactical 
decision and each secondary enemy and friendly kill decision made for 
the entire sample. Means and standard deviations of these variables 
and decisions were then computed for each subset of the sample defined 
by biographical data groupings, ioC. , all Commanders, all Lieutenant 
Commanders, all Lieutenants, all Lieutenants (junior grade), all pilots 
with more than 4000 hours, all with less than 4000 and more than 3000, 
all with less than 3000 and more than 2000, all with less than 2000 and 
more than 1000, all with less than 1000, all pilots with over 300 missions, 
all with less than 300 and more than 200, all with less than 200 and 
more than 100, all with less than 100 and more than one mission and 
those pilots with no combat missions. Likewise, the mean and standard 
deviation of each decision was computed for the subset of the sample 
defined by holding each of the situational variables constant at one of 
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TABLE I 



DESCRIPTION OF RESPONDING FIGHTER PILOTS 



Rank: 


LT(jg) 


LT 


LCDR 


CDR 


Number: 


1 


21 


10 


4 


Hours: Mean 


450.0 


1394.29 


2675.50 


3825.00 


S.D 


0 


438.54 


553.69 


466.61 


Missions: Mean 


0 


96.52 


163.40 


201.75 


S.D. 


0 


74.66 


72.17 


106.81 



Hours: 


0-999 


1000-1999 


2000-2999 


3000-3999 


4000+ 


Number: 


5 


17 


6 


6 


2 


Rank^: Mean 


2.80 


3.14 


3.67 


4.33 


5.0 


S.D. 


.01 


.88 


.47 


.47 


0 


Missions: Mean 


0 


127.07 


111.83 


242.00 


106.50 


S.D. 


0 


87.12 


54.78 


69.20 


36.64 



Missions: 


0 


1-99 


100-199 


200-299 


300+ 


Number: 


6 


7 


13 


8 


6 


Rank^; Mean 


2.83 


3.38 


3 . 54 


3.71 


4.50 


S.D. 


.37 


.70 


.63 


.70 


.50 


Hours: Mean 


763.33 


2150.00 


2065.77 


2378.57 


3250.00 


S.D. 


177.87 


971 .43 


870.87 


832.28 


50.20 



^Ensign = 1, LT(jg) = 2, LT = 3 , LCDR = 4, CDR = 5 
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its values and then the other while the remaining situational variables 
were free to assume either of their two values . 

In attempting to find a relationship between the biographical and 
situational variables and the decisions reached, correlation coefficients 
were computed between these variables and decisions. These correlation 
coefficients were computed for the entire sample, for each subset de- 
fined by the biographical groupings, and for each respondee. These 
findings were presented to a panel of Naval aviators who were asked for 
their opinions and suggestions. These aviators are described in Appendix 
C. From their suggestions new variables made up of combinations of 
the original six variables were added to the data base and the same 
analysis again undertaken. This cycle was repeated until the panel con- 
sidered the results to be reasonable. 

In an attempt to distinguish any discernible differences between 
the decisions reached for each biographical subset within the rank, hour 
and mission groupings, a discriminant analysis was performed. In this 
analysis, a linear function for each subset within a group is computed 
using the subset characteristic (specific rank, hours grouping, or 
mission grouping) as the dependent variable and the decisions reached 
as the independent variables. These functions are then evaluated for 
each case and the hypothesis that the subset means are the same is 
tested [9] . 
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In an attempt to predict a pilot or group of pilots' decisions, a. 



sequencc of multiple linear equations computed using stepwise multiple 
regression techniques were also produced for the entire sample and each 
subset. At each step in this regression procedure, one situation variable 
is added. The variable added is the one which makes the greatest re- 
duction in the error sum of squares (the variable with the highest partial 
correlation with the dependent variable at that step) . The production 
stops when the error sum of squares decreases less than .001 or until 
all variables are exhausted [10] . The correlation coefficient between 
these equation predictions and the actual decision reached was then 
computed for each situation examined. These equations and correlations 
were then presented to the panel of aviators for their evaluation. 
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III., pr..qm.TS OF DATA JM^M^ 



A . VARIABLES ADDED AND DELETED 

Since bearing and heading , although guantitative items of infor- 
mation Shown to be important in the first survey, did not represent con- 
tinuums that could be conveniently statistically manipulated, both 
were deleted in favor of a new variable. The reason this was done is 
apparent when evaluating, for example, bearings of OOr and 359”. 

Although separated by only 2” on a compass rose, a guantity evaluated 
by the panel of aviators as negligible, the values are separated by 

357” which is not negligible. Also, the two variables' effects on a 

Pilot's evaluation appeared to be interactive. An enemy directly behind 

a friendly (bearing 180”) was evaluated differently if the enemy was 

pointed at the friendly (heading 360”) or pointing away (heading 180”). 

This pointing information is not directly available to the pilot from either 

his cockpit displays or GCI input. Yet the pilot does mentally integrate 

ihe bearing and heading Information into pointing information. Conse- 

guently, a function of both bearing and heading was created. This 

function, called "danger angle," is described by the following eguationt 

Danger (bearing, heading) = (sin (bearing)/! sin (bearing) 1 ) 

X [bearing - (heading + ISO) modulus 360] . 

Similarly, although pilots considered range and fuel -remaining as 
important variables independently, the aviator panel believed that a 
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link existed between these two factors. For instance, a fighter pilot's 



behavior in a situation of long range with low fuel remaining would 
probably be different from that under a situation of short range and a 
large amount of fuel remaining. Again, this would be a case of pilot 
integration of a cockpit display (fuel remaining) and a GCI input or 
radar display (range) into a range per fuel state factor. Consequently 
a range-fuel ratio (MPG) was created as described by the following 
equation: 

MPG (range, fuel remaining) = range/fuel remaining. 

As shown in the first mail-out questionnaire results, a relationship 
between the enemy's speed and heading and friendly speed and heading, 
or closure rate is considered by fighter pilots (See EX PE PJ MENTAL 
PROCEDURE DETERMINING DECISION STPvATEGY) . Again, pilot inte- 
gration of cockpit and GCI inputs appears to occur: the pilot projecting 
the parallel enemy speed vector into his ov/n. As a result, a speed 
differential function was created for the tactical scenario: 

Speed (heading, bearing) = 500 x cos (heading) - (cos (bearing)/ 

J cos (bearing) j ) x 450. 

In both the speed and danger equations, a value of +1 was assigned to 
the ratio of the trigonometric function and its absolute value when that 
function was equal to zero. 
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B. CORRELATIONS 

The primary tactical decision made by the subjects was assumed 
to be ordinally quantifiable as indicated below; 

(1) Pilot has no choice, an engagement would occur. 

(2) Pilot has a choice and would elect to engage the enemy. 

(3) Pilot would not engage the enemy . 

The secondary decision was in two parts, number of enemy losses pre- 
dicted and number of friendly losses predicted if an engagement were 
to occur. 

The biographical data's correlation with the decision is shown in 
Table II for all pilots . 



TABLE II 

CORRECTION OF BIOGRAPHICAL DATA WITH PRIMARY 
AND SECONDARY DECISIONS 





Tactical 


Enemy 


Friendly 




Decision 


Kills 


Kills 


Rank 


-.0392 


.0541 


.2195 


Hour 


-.0394 


.0628 


.1685 


Missions 


-.0333 


.0801 


. 1265 



The lack of correlations significantly different from zero is striking. 
Only with the relationship of rank to friendly kills predicted can a 
coefficient of correlation be found of any sizeable magnitude (+.2195). 
Discriminant function analysis revealed no significant differences 
between the pilots when they were categorized by rank, hour or mission 
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with respect to the decisions they made (at a significance level of 
< . 05) (See Appendix D) . The panel of aviators considered this (post 
hoc) to be a result of standardized training and operational squadron 
procedures. They also expected a positive correlation between rank 
and expected friendly kills as a result of more realistic evaluation 
techniques acquired with time spent in operational billets. A decline 
in youthful optimism as one ascends the rank and age structure was also 
expected to enhance this postulated positive correlation. Appendix F 
contains the means and standard deviation of the decisions reached by 
the biographical groupings. 

The situational variables’ conrelations with the decisions are 
shown in Table III. 



TABLE III 



CORRELATIONS OF SITUATIONAL VARIABLES WITH 
PRIMARY TACTICAL DECISIONS AND SECONDARY 
EXPECTED ENEMY AND FRIENDLY KILL DECISION S 



Tactical 

Decision 



Enemy 

Kills 



Friendly 

Kills 



MPG 

Speed 

Danger 

Fuel Remaining 
Range 

Number of Enemy 
Rules of Engagement 



.4259 

.2559 

.2864 

-.1708 

.4209 

.1223 

-.0760 



-.2087 

-.2339 

-.2857 

.3400 

-.0163 

.0231 

.2287 



-.1237 

.0553 

-.0201 

-.0203 

-.1746 

.2447 

-.0512 
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The panel of aviators, however, felt a chronological relationship 
existed between the decisions, that is, the pilot evaluated the situation 
as a whole and then reached his primary tactical decision. After that 
decision had been reached, and with that result kept in mind, it was 
thought that the pilot then made his secondary decisions. Suggestions 
by the author that the decisions of kill possibilities would precede and 
influence the tactical decision were strongly rejected by the panel as 
unrealistic from their ov/n experiences. Table IV shows the correlation 
between the primary and secondary decisions. 

TABLE IV 

CORRELATION BETWEEN PRIMARY AND 
SECONDARY DECISIONS 

Enemy Kills Friendly Kills 

Tactical Decision -.4247 -.2307 

The reader should note that the biographical variable of rank is 
the third most important variable when predicting secondary friendly 
kills. 

C. VARIABLE MANIPULATION 

The effects of holding specific situation variables constant on the 
decisions made is shown in Table V. MPG was not held constant since 
that would imply a simultaneous manipulation of range and the fuel- 
remaining variable. 
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The most striking phenomena evident in Table V is the ratio 



between predicted enemy kills and predicted friendly kills of over four 

to one. As explained by the senior member of the panel of aviators 

after observing this phenomena: 

"A pilot would be a fool to go into combat is he didn't think 
the odds were stacked heavily in his favor." 

The inverse relationship evident in this table between range and 
friendly kills was considered by the aviators to be a function of aircraft 
type. The F-4 was considered by them to be more of a weapons platform 
than a classical dog-fighter while the MIG-21, a smaller and more 
maneuverable airplane, is better suited for close-in fighting. The 
shorter range and, consequently, time available to the F-4 weapons 
systems decreases some of this system's advantages over a MIG-21 
resulting in a greater enemy opportunity to attack the friendly forces . 
Interesting conjectures can be made about the results that might be 
observed if the friendly plane were an F-8, a true dog-fighter. . .The 
slight increase in enemy kills with decreasing range which is evident 
in Table V was felt by the panel of aviators to be the result of an in- 
creased friendly missile kill probability. 

The pushing of the tactical decision to the "must engage" pole 
with shorter ranges was felt to be an indication that a pilot naturally 
considers danger more imminent as the enemy closes; the shorter ranges 
decreasing the protector's options . 
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TABLE V 



VARIABLE MANIPULATION -AVERAGE 
VALUE PER SITUATION 



Variable 


Value Tactical Decision 


Enemy Kills 


Range 


4 


1.66 


1.09 




20 


2.32- 


1.05 


Rules of 


Missile Free 


1.93 


1.31 


Engagement Eyeball 


2.05 


.84 


Number of 


2 


1.89 


1.05 


Enemy 


6 


2.09 


1.10 


Speed 


0 


1.71 


1.42 




0 


2.28 


.73 


Danger 


0 


2.03 


.94 




0 


1.83 


1.19 


Fuel 


Remaining 1000 


2.12 


.73 




2500 


1.86 


1.42 


All Free 


Mean 


1.99 


1.07 




S.D. 


.79 


1.01 



Friendly Kills 

.37 
. 15 

.23 

.29 

.11 

.41 

.24 

.28 

.23 

.29 



.27 

.25 

.26 

.62 
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The differences displayed in the decisions as rules of engagement 
are varied were also felt by the panel to reflect aircraft type. The F-4 
is at its best when able to deliver the long range, unsuspected missile 
under missiles free conditions. A 40% drop in predicted enemy kills can 
be seen in Table V as the rule of engagement assumes the eyeball con- 
dition. Waiting for ranges to shorten for eyeball identification means 
losing the advantage of the long range punch. The decreased range 
forced on the pilot shows in Table V, as before; in an increase in pre- 
dicted friendly kills but not, surprisingly, in a tactical evaluation pushed 
towards the "must engage" pole. 

The greatly increased number of predicted friendly kills when the 
number of enemy is high, as shown in Table V, is considered to be a 
direct reflection of superiority in aircraft numbers . No matter how 
effective a protector may be, once the protectors are saturated, some 
of the enemy will reach the vulnerable friendly forces. Also, the more 
enemy present the more enemy the friendlies may destroy. The tactical 
decision's migration towards the "not engage" pole was felt by the panel 
to be a sign of caution by the friendly in light of a large enemy force; 
the gains of engagement being more than offset by the possible friendly 
losses . 

The speed differences between the friendly and enemy forces 
appear to push the primary tactical decision towards the two poles of 
engage or not engage. A large speed differential coupled with opening 
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ranges implies, according to the panel, a friendly decision of leaving 
the friendly forces exposed for a long chase or staying with the friendlies 
and not engaging the enemy. The high closure rate, however, would 
force the friendly to engage but, as shown in the extremely high predicted 
enemy kills in Table V, would place the enemy in the F-4's most optimal 
firing envelope . 

The enemy pointing at the friendly forces places the friendlies in 
the enemy's best firing envelope and, as shown in Table V, seem to 
force the friendly tactical decision to the defensive "must engage" pole. 
As a result, a rise in both predicted enemy and friendly kills can be 
seen. The enemy pointing away from the friendlies with its implied 
opening ranges drives the tactical decision to the "not engage" side 
with concurrent drops in both predicted kills . 

The low enemy kills and "not engage" decision evident with low 
fuel state was interpreted by the panel as a sign for caution for the 
friendly pilot. Friendly kills rise in this low fuel state not only from 
enemy attacks but from destruction due to fuel exhaustion in combat. 
Higher fuel states give the pilot a greater margin in combat for increased 
maneuvering and chasing, resulting in the increased predicted enemy 
kills and greater friendly protection. 
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D. MULTIPLE REGRESSION 



From the multiple regression analysis a ranking of situational 
variable importance was made for all pilots and for each rank, hour and 
mission group. Table VI gives these rankings for all pilots and for the 
rank groupings. Appendix G contains the hour and mission rankings. As 
can be seen, MPG or range, one a function of the other, play an important 
role in the tactical decision evaluation. Danger, except in the case of 
Gommander where it switches position with speed, is next most important 
followed by speed. 

In the importance of variables for secondary decision evaluation, 
the tactical decision is the most important in evaluating enemy kills 
while the number of enemy is the only variable more important in evalu- 
ating friendly kills. This may reflect the pilot's integration of his 
primary decision in his next evaluation, i.e. , the pilot’s "mental set" 
before actually making his next evaluations. 

From this multiple regression analysis predictor equations were 
constructed. Described in Appendix E, the equations were used in 
predicting the decisions made on the decision survey. The obtained 
multiple correlations are described in Table VII and Appendix H. 
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TABLE VT 



VARIABLE RANKINGS FOR ALL PILOTS AND 
RANK GROUPS IN DECISIONS REACHED 



A. Tactical Decision 

ALL 



MPG 1 

Danger 2 

Speed 3 

Range 4 

Number of Enemy 5 

Rules of Engagement 6 

Fuel Remaining 7 



B . Enemy Kills 

ALL 



Tactical Decision 1 

Fuel Remaining 2 

Rules cf Engagement 3 

Danger 4 

Speed 5 

Range 6 

Number of Enemy 7 

MPG 8 

C . Friendly Kills 

ALL 

Number of Enemy 1 

Tactical Decision 2 

Speed 3 

Rules of Engagement 4 

Fuel Remaining 5 

Danger 6 

Range 7 

MPG 8 



CDR 

7 


LCDR 


LT 

L 


LT(jg) 

7 


3 


2 


2 


2 


2 


3 


3 


6 


1 


1 


5 


1 


4 


4 


4 


4 


6 


6 


6 


5 


5 


5 


7 


3 



CDR 


LCDR 


LT 


LT(jg) 


1 


1 


1 


1 


4 


3 


2 


2 


6 


4 


3 


8 


5 


2 


4 


4 


8 


5 


5 


3 


2 


6 


6 


6 


3 


7 


7 


5 


7 


8 


8 


7 



CDR 


LCDR 


LT 


LT(jg) 


1 


2 


1 


1 


2 


1 


2 


4 


4 


4 


3 


3 


3 


7 


4 


5 


8 


5 


5 


8 


5 


8 


6 


6 


7 


3 


7 


2 


6 


6 


8 


7 
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TABLE VII 



CORRELATION OF PREDICTED VALUES 
FOR DECISIONS AND ACTUAL DECISIONS 
MADE BY ALL PILOTS AND RANK GROUPINGS 





ALL 


CDR 


LCDR 


LT 


LT(jg) 


Tactical Decision 


. 6032 


.5794 


.6236 


.6101 


. 7568 


Enemy Killed 


.6027 


.5859 


.5654 


.6527 


.8193 


Friendly Killed 


.4015 


.4391 


.5299 


.3732 


.5284 



It. appears that the equations are fairly successful for the groups studied 
in predicting their tactical and enemy kill decisions. 

In an attempt to secure a standard of comparison for these equations, 
the panel of aviators was used to predict one of the biographical group's 
decisions. The members of the panel, who were associated with the 
data at some length, considered themselves to be closest to the 
Lieutenant Commander group in rank, hours, and missions. All of the 
Lieutenants on the panel were in the current Lieutenant Commander 
selection zone. They attempted to predict the Lieutenant Commander 
group's decision rather than only making their own evaluations of the 
situations. The following correlations between the panel's predictions 
and the actual decisions made by the Lieutenant Commander group were 
found: 



Tactical Decision 


.5863 


Enemy Killed 


.5200 


Friendly Killed 


.4327 
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In essence, the predictor equations fared better than the predicting 



panel of aviators (See Table VII) . 
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IV.. EXPERIMENTAL PROCEDURE-ARTIFICIAL INTELLIGENCE APPLICATION 



One writer [11] has described and defined artificial intelligence in 
the following way: 

"A human being can think, learn, and create because the program 
his biological endowment gives him, together with the changes in 
that program produced by interaction with his environment after 
birth, enables him to think, learn, and create. If a program thinks, 
learns, and creates, it will be by virtue of a program that endows 
it with these capacities. Clearly this will not be a program - any 
more than the human's is - that calls for highly stereotyped and 
repetitive behavior independent of the stimuli coming from the 
environment and the task to be completed. It will be a program 
that makes the system's behavior highly conditional on the task 
environment - on the task goals and on the clues extracted from 
the environment that indicate v/hether progress is being made towards 
those goals. It will be a program that analyzes, by some means, 
its own performance, diagnoses its failures, and makes changes 
that enhance its future effectiveness." 

With the goal of the last sentence above in mind, the panel of 
aviators, all of whom have had student aviator instructor billets, 
described a student pilot's behavior in interaction with his instructor. 
Once past the basic ground schools, flight training appeared to entail 
corrective instruction of the student's cockpit actions. The corrective 
instruction can be broken down into three general areas: 

(1) The student did the opposite of what he should have done. 

(2) The student did too much of what he should have done. 

(3) The student did not do enough of v^hat he should have done. 
For example; (1) pushing the stick to the left vice the right, (2) applying 
too much throttle, and (3) not using enough flaps. Such mistakes as 
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forgetting to lower the landing, gear were considered as combinations or 



extremes of the tliree . 

The student's response to the three corrections were: 

(1) Reversing the sign or direction of what was incorrectly done. 

(2) Lessening the magnitude of what was incorrectly done. 

(3) Increasing the magnitude of what was incorrectly done. 

Also, as the student pilot gains more experience his confidence level in 
himself and aircraft rises. His corrections grow smaller in magnitude 
resulting in a smoother development. As stated by one of the aviators 
on the panel: 

"Any movement which is not smooth and gentle in an aircraft is 
probably an incorrect movement." 

In advanced Naval pilot training where such topics as tactics and 
gunnery are taught, the three instructive areas are thought to still exist. 
However, at this level the pilot is developing his evaluative skills 
rather than his motor skills. Since multi-level decisions are involved, 
the student is not allowed to proceed to the next level until the present 
level decision or action is correct. Figure 3 displays this macro-level 
education concept. 

This concept, if captured in a program, would enable the program 
to analyze its performance and diagnose its failures (instructor correction) 
and make changes to enhance its future effectiveness (student modifi- 
cation) . This program could show some attributes (confidence and 
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Figure 3 

MACRO-LEVEL PILOT EDUCATION 



A: Present situation to student. 

B: Student evaluates situation and makes primary decision. 

C: Is decision correct? 

D: Student gains confidence and experience. 

E; Student makes secondary decisions. 

G: Student modifies own evaluation techniques. 
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experience, learning, fallibility) that, if: witnessed in a living organism, 
would certainly be termed intelligent. Hence, an artificial intelligence 
approach to this macro-level pilot learning concept was initiated. 

Certain restrictions were placed on this artificial intelligence 
programming attempt. Since possible graphics adaptations were en- 
visaged, memory size limitations were desired. The available graphics 
terminal at Naval Postgraduate School is an AGT-10 with limited memory. 
Also, as brought out by the panel of aviators, a pilot does not consciously 
search through his memory looking for a situation or solution but rather 
appears to act at a level of perceptual decision making [7]. Likewise, 
the panel considered the classical tree searching and minimaxing pro- 
cedures as unrealistic for this problem. Computer run time was to be 
as short as possible for both the practical consideration of program turn 
around time and because of the desire for instantaneous decisions as in 
high speed air-to-air combat. Consequently, a polynomial evaluation 
with adaptation of the coefficients technique was chosen along the lines 
of Samuel's checker player [2] . The coefficient would, in essence, act 
as a quasi-memory, reflecting change but not, necessarily, history. 
FORTRAN IV was chosen for the program language over ALGOL and LISP 
because of future graphics applications where no ALGOL or LISP compilers 
would be available. 
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The "instructor" would be one of the pilot groups for which a 



predictive equation existed . The instructor's memory would contain 
the predictive equation and all of the decisions made concerning the 
64. situations presented in the second survey. The "student" would 
contain a predictive equation, either from actual data analysis, or a 
dummy one. It would have no decision history memory. 



A. PROGRAM DESCRIPTION 

The following matrices and vectors of interest are used throughout 
the program; 

(1) sit (i, j) i = 1 , 64; 3 = 1 , 6 

This matrix contains the six situational variables of the 
second array: 

j = 1 - fuel remaining, 
j = 2 - range , 
j = 3 - bearing, 
j = 4 - heading , 

3 = 5- number of enemy, 

3 = 6- rules of engagement. 

They are presented in the 64 different situations, i going 
from 1 to 64 . 

(2) mem (i,j) i = 1 , 64; j = 1 , 3 

This matrix contains the three decisions made by the 
instructor over the 64 situations: 

j = 1 - primary tactical decision, 
j = 2 - secondary enemy kills decision, 
j = 3 —secondary friendly kills decision. 
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(3) prico (1) i = 1 , 8 

This vector contains the instructor's primary tactical 
decision predictive equation coefficients for the variables 
below; 

i = 1 - a constant 
1=2 - fuel remaining 
1= 3 —range 
1=4- number of enemy 
i = 5 - rules of engagement 
1—6 - danger 
1=7- speed 
i = 8 - MPG 

/ 

(4) startp (i) i = 1 , 8 

Completely analogous to prico, this vector contains the 
student's primary tactical decision coefficients. 

(5) secoen (i) i = 1 , 9 

This vector contains the instructor's secondary enemy kills 
decision predictive equation coefficients for the same 
variables as in prico. When i = 9, the coefficient is for the 
primary tactical decision reached. 

(6) secofr (i) i = 1, 9 

This vector is the same as secoen but for the secondary 
friendly kills decision predictive equation. 

(7) starse (i) i = 1 , 9 

Completely analogous to secoen, this vector contains the 
student's secondary enemy kills decision coefficients. 

(8) starsf (i) i = 1 , 9 

Completely analogous to secofr, this vector contains the 
student's secondary friendly kills decision coefficients. 

(9) kor (i) i = 1 , 8 

This vector is the vehicle for the instructor's correction of 
the student's startp coefficients. 

(10) kore (i) i = 1 , 9 

This vector is the vehicle for the instructor's correction of 
the student's starse coefficients. 
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( 11 ) 



korf (i) i = 1 , 9 

This vector is the vehicle for the instructor's correction of 
the student's starsf coefficients. 



1 . Main Program 

The main program contains, in essence, the previously 
described macro-level pilot education concept. The 64 situations are 
read in (sit(i,j)) followed by the instructor's decisions over these sit- 
uations (mem(i,j)) followed by the instructor's predictive equations 
(prico(i), secoen(i), secofr(i)). This constitutes the instructor's memory 
and expertise, his references and standards. Next, the student is 
brought into the scene by the reading of his starting evaluation equations 
(startp(i), starse(i), starsf(i)), these data representing the starting 
student at the beginning of the interactive instructor-student-situational 
process . 

Next, for each of 900 trials, one of the 64 situations is picked 
at random and presented to the student. After calculating danger, speed 
and MPG; a skill hopefully developed through previous training, but 
probably never applied, the student makes the primary tactical decision 
for situation I: 

IDECIS = startp (1) + .5 + startp (2) x sit (1,1) 

+ startp (3) X sit(I,2) + startp (4) x sit (I, 5) 

+ startp (5) X sit (1, 6) + startp (6) x danger 
+ startp (7) X speed + startp (8) x MPG. 

A subroutine INSTRU is now called. This is equivalent to the 
instructor grading the student by saying "You've acted in the wrong 
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direction"; "too much"; "too little"; or "correct." From this grade the 
student is either sent on to make his secondary decisions (correct) or 
his primary evaluation equation is sent to KORECI subroutine for modi- 
fication. If the student's grade is "not correct," upon returning from 
modification he returns to the beginning to have another situation 
presented without attempting to make a secondary decision. If no modi- 
fication is needed, the student's experience and confidence level is 
increased and the secondary decisions are made for situation I: 

IIEK = starse (1) + .5 + starse (2) x sit(I,l) 

+ starse (3) x sit(I,2) + starse (4) x sit(I,5) 

+ starse (5) x sit(I,6) + starse (6) x danger 
+ starse (7) x speed + starse (8) x MPG 
+ starse (9) x IDECIS (enemy kills) , 

IIFK = starsf (1) + . 5 + starsf (2) x sit (1 , 1) 

+ starsf (3) x sit(I,2) + starsf (4) x sit(I,5) 

+ starsf (5) x sit (I, 6) + starsf (6) x danger 
+ starsf (7) x speed + starsf (8) x MPG 
+ starsf (9) x IDECIS (friendly kills) . 

As in INSTRU, these decisions are graded by subroutine GRADER. 
These grades are considered after which either the student's equations 
are modified by KOREII or KORFII or his experience and confidence levels 
are increased. In either case, the student is sent back to the beginning 
for a new situation. The grading and correcting of the secondary enemy 
kill and friendly kill decisions are handled independently, hence 
correction may take place in one but experience and confidence may rise 
in the other . 
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Subroutine INSTRU 



2 .. 

INSTRU first determines if it agrees with the student’ s 
primary tactical decision by checking the instructor's memory for the 
same situation (mem(1, 1)) . If the instructor and the student agree this 
fact is immediately returned to the main program. If there is no agree- 
ment, the instructor undertakes a coefficient by coefficient comparison 
of its own predictive equations (prico) with the student's evaluation 
equation (startp) for the primary decision. First compared as to sign 
and then as to magnitude, a grade is sent back to the main program for 
each coefficient; e.g. , wrong sign, too small, too large, or, agrees 
with instructor's coefficient. The coefficient comparison is undertaken 
only when the student's primary decision does not agree with the 
instructor' s . 

3 . Subroutine GRADER 

Subroutine GRADER is essentially the same as subroutine 
INSTRU but deals with the grading and coefficient comparison of the two 
secondary decision equations. The two secondary decisions are handled 
independently. 

4 . Subroutine KORECI 

Subroutine KORECI handles the modification of the student's 
equation for his primary tactical decision. A student's attempted cor- 
rection of the same error is not necessarily stable in magnitude. 
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especially when he is first beginning to learn to fly. As a student's 
experience and confidence grows, hovyrever, the modifications become 
more consistent and decreasing in magnitude. Consequently, an 
experience modification factor, ZZ, is set. Previous runs had shown 
that multiplicative modifiers differing more than about .11 from 1 caused 
an erratic and sudden convergence of the student's coefficient to that 
of the instructor's. Therefore, it was decided to limit the modification 
factor to the closed interval [9 , 1.1]. 

The experience modification factor is set by taking a uniformly 
distributed random number from the open interval (0,1). A different 
number is chosen each time KORECI is called. This number is then 
divided by the sum 10 + .lx experience and confidence level. This 
experience and confidence level (X) is simply the number of correct 
primary tactical decisions that have been made. Hence the experience 
modification factor (ZZ) is in the open interval (0, .1). Adding or sub- 
tracting ZZ from one produces a modification factor in the open interval 
(.9,1.1) that approaches unity. For example; a starting random number 
of .5 with no experience and confidence gives a modification factor of 
.95 or 1.05, a starting random number of .5 with an experience and 
confidence level of 200 gives a modification factor of .98 or 1.02, and 
a starting random number of .9 with an experience and confidence level 
of 1000 gives a modification factor of . 99 or 1.01. 
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After the experience modification factor is set, the student's* 



coefficients are checked to see if the variable paired with that co- 
efficient is even being considered, i.e. , is the coefficient zero? If 
so, subroutine NOVICE is called, which means that more intensive 
instruction is given, or, a return is made to a more elementary student 
state. If not, the coefficients' gradings are examined and the co- 
efficients are modified according to the results of these gradings . If 
a wrong sign is indicated, the student's coefficient is multiplied by -1. 
If the coefficient is too great in magnitude, the student's coefficient 
is multiplied by the modification factor 1-ZZ. If the coefficient is too 
small, the modification factor is 1+ZZ. A coefficient determined by 
INSTRU as being correct is not modified. 

5 . Subroutine KOREII 

Subroutine KOREII is essentially the same as subroutine 
KORECI but handles the secondary enemy kill decisions. The experience 
and confidence level is the total number of correct secondary enemy kill 
decisions made by the student. 

6 . Subroutine KOREII 

Subroutine KOREII is identical to subroutine KOREII but 
handles the secondary friendly kill decisions. The experience and con- 
fidence level is the total number of correct secondary friendly kill 
decisions made by the student. 
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7., 



Subroutine NOVICE 



The object of this subroutine is to instruct the student to 
consider a variable that, up to this time, was not being considered 
(coefficient of zero) . Since data had revealed that out of 330 coefficients 
considered in the analysis of all pilots and rank, hours, and mission 
groups, 329 fell into the interval (-30,30), a uniformly distributed 
random number is selected from the interval (-30,30) each time NOVICE 
is called. This is then given the sign of the instructor's coefficient 
for that variable and assigned to the student's coefficient in lieu of the 
previous zero value. 

8 . Subroutine RANDOM 

This subroutine, a slightly modified version of IBM's RANDU 
[12], simply generates a uniformly distributed random number in the 
inteiwal (0,1). 

(All routines can be found in the Computer Printout Section 
of this thesis) . 

B. RUNNING THE PROGRAM 

The program was run with the average pilot for the entire sample, 
the statistically average pilot from each rank group, the statistically 
average pilot from each hour group and the statistically average pilot 
from each mission group as instructors with the same groups' predictor 
equations as starting coefficients for the students. Therefore, each 
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instructor was run with all of these average pilots as students. As can 



be seen in Figures 4 through and Table VIII, little differences exist 
between the instructor-student performances. 

As a result of these runs, a synthetic beginning student was con- 
structed with all starting coefficients set to zero. Using this beginner 
model as a student, the program was rerun with: 1) the average pilot 

for the entire sample, 2) the Commander group, 3) the 4000-plus hour 
group, and 4) the 300-pius mission group as instructors. The latter 
three groups were chosen for their expertise under the assumption that 
a pilot who has attained higher rank, flown 4000 non-fatal hours or 
survived 300 missions must possess some expertise in flight. The 
results obtained from this "beginner" student run were plotted and 
compared to those of the average student (See Figures 4-15, 16-27 and 
Table VIII) . These comparisons and plots were then presented to the 
panel of aviators for their comments and criticisms. 
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TABLE VIII 



RESULTS OF PROGRAM RUNS WITH TV\^0 STUDENTS 
AND FOUR INSTRUCTORS 



Average Pilot as Student: 








Time* to Make First Correct Decision 








Instructor: Average Pilot 


CDR 


4000+ 


300 + 


Primary Decision 5 


5 


5 


5 


Secondary Enemy Kills 5 


5 


5 


5 


Friendly Kills 5 


5 


5 


5 


Time* to Attain 90% of Final Correct Percentage After First Correct 


Decision 








Instructor: Average Pilot 


CDR 


4000+ 


300 + 


Primary Decision 5 


62 


40 


96 


Secondary Enemy Kills 19 


39 


8 


73 


Friendly Kills 5 


57 


8 


95 


Final Correct Decision Proportions 








Instructor: Average Pilot 


CDR 


4000+ 


300+ 


Primary Decision . 85 


.67 


.87 


.83 


Secondary Enemy Kills . 72 


.63 


.60 


.55 


Friendly Kills .94 


.71 


.62 


.64 


Synthetic Beginner as Student: 








Time* to Make First Correct Decision 








Instructor: Average Pilot 


CDR 


4000+ 


300+ 


Primary Decision 200 


221 


199 


206 


Secondary Enemy Kills 129 


194 


172 


149 


Friendly Kills 157 


177 


161 


177 


Time* to Attain 90% of Final Correct Percentage After First Correct 


Decision 








Instructor: Average Pilot 


CDR 


4000+ 


300 + 


Primary Decision 40 


47 


48 


83 


Secondary Enemy Kills 13 7 


40 


32 


104 


Friendly Kills 56 


111 


72 


225 


Final Correct Decision Proportions 








Instructor: Average Pilot 


CDR 


4000+ 


300 + 


Primary Decision .84 


.65 


.86 


.83 


Secondary Enemy Kills . 72 


.61 


.60 


.55 


Friendly Kills .93 


. 71 


.60 


.61 


*Time is in units of trials for the primary decision 
primary decisions for the secondary decisions. 


and in units 


of correct 
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V.. RESULTS OF PROGRAM RUNS 



A. AVERAGE PILOT AS STUDENT 

For the case of the average pilot for the entire sample as the 
student, a successful evaluation of the situations occurred early in the 
runs, i.e. , within the first five trials for all instructors. As can be 
seen in Table VIII, the average pilot instructing the average pilot had 
the highest overall average of correct decisions: 85% for the primary 

tactical decision, 72% for the secondary enemy kill decision and 94% 
for the secondary friendly kill decision. Naturally, the student's ending 
coefficients were the same as the student's starting coefficients since 
they are identical to the instructor's. The extremely high percentage of 
secondary friendly kill decisions is puzzling in light of the low corre- 
lation coefficient of .4015 between the predictor equation with actual 
secondary kill decisions made. Varying the random number seed changed 
that percentage negligibly. An explanation may lie in the fact that the 
rounding of the program decision equations to the nearest integer improved 
the accuracy of the prediction with respect to the correlation observed. 

For example, a predicted value of 1.49 friendly kills would be rounded 
to one friendly kill by the artificial intelligence program. Assuming the 
actual friendly kill decision to be one friendly kill, the artificial intel- 
ligence program vv^ould show a higher correlation than the straight 
predictor equations . 
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On the assumption that a student, upon reaching 90% of his final 



performance, is near matriculation, the time for his running percentage 
of correct decisions to reach 90% of his final percentage after his first 
correct decision was calculated. The running percentage of correct 
decisions reached 90% of the final percentage within five trials for the 
primary tactical decision, five correct primary decisions (a correct 
primary decision corresponds to a trial for the secondary decision) for 
the secondary friendly kill decision and 19 correct primary decisions for 
the secondary enemy kill decision. 

The Commander instructor, as can be seen in Table VIII, whose 
predictor equations differed the most from the average pilot's, shov/ed 
the lowest final percentage of successful decisions except in the case 
of the secondary friendly kill decision; 67% for the primary tactical 
decision, 63% for the secondary enemy kill decision and 71% for the 
secondary friendly kill decision. The high friendly kill decision per- 
centage may be a reflection of a rank-friendly kill correlation witnessed 
in the data analysis (See Table II) . The running percentage of correct 
decisions reached 90% of the final percentage within 62 trials for the 
primary decision, 39 correct primary decisions for the enemy kills and 
57 correct primary decisions for the friendly kills. These longer learning 
times are felt to be reflections of the instructor and student coefficient 
differences . 
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The 4000-plus hour instructor, as can be seen in Table VIII, 



closely matched the average pilot instructor in final percentage of 
primary tactical decisions with 87% and neared the Commander instructor 
in. final percentage of secondary enemy kill decisions with 60%. The 
4,000-plus instructor displayed the lov^^est secondary friendly kill decisions 
with 62%. The time for running percentages to reach 90% of the final 
percentages for this run were moderately low at 40 trials for the primary 
decision, and eight correct primary decisions for both secondary decisions. 
The low number of primary decisions correct needed to reach 90% of the 
secondary percentages is a reflection of the lower final percentages. 

The 300-plus mission instructor, as shown in Table VIII yielded 
final percentages of 83%, 55% and 64% for the primary and secondary 
enemy and friendly kill decisions, respectively. Running percentages 
reached 90% of final percentages within 96 trials for primary decisions, 

73 correct primary decisions for the secondary enemy kill decision and 
95 for the friendly kill decision. These longer learning times are felt 
to be a result of the modification factor (ZZ) rather than a reflection of 
lower percentages or instructor student differences. Although no such 
run was made, the panel of aviators felt that a changed random number 
seed would show reduced learning times . 

However, as can be seen on the graphs of total decisions right 
versus total trials or total primary decisions correct (See Figures 4-15), 
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the results are similar and nearly linear for all cases. The panel of 
aviators felt this pattern to be consistent with their own experience, 
i.e. , as a result of standardization of procedures and training, different 
instructors teaching similar pilots v/ould have, for the most part, similar 
results.. They also considered the student, the average pilot from the 
sample, as more experienced than the actual student encountered in a 
flight syllabus. However, in a refresher or replacement training situation 
where the student is experienced, such rapid attainment of consistent 
results as witnessed in the graphs is felt by the panel to be typical. 

B. SYNTHETIC BEGINNER AS STUDENT 

The synthetic beginner student, with all starting coefficients set 
to zero displayed a radically different behavior than that of the average 
pilot (See Table VIll) . At the end of 900 trials, all the running per- 
centages for the last 100 decisions were equal to the final percentages 
witnessed when the average pilot was the student. Inspection of the 
runs revealed the interesting phenomenon of long error periods followed 
by rapid learning to a steady percentage. 

With the average pilot as instructor, the first correct primary 
tactical decision was reached on the 200th trial. The first correct 
secondary enemy kill decision was not reached until after 129 correct 
primary decisions had been made, the first correct secondary friendly 
kill decision was not reached until after 157 correct primary decisions. 
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Figure 4 
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Figure 5 
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Figure 6 
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Figure 7 
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Figure 9 
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Figure 10 



AVERAGE STUDENT-4000 HOUR INSTRUCTOR 




58 



Figure 11 
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Figure 13 



AVERAGE STUDENT-300 MISSION INSTRUCTOR 
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Figure 14 



AVERAGE STUDENT-300 MISSION INSTRUCTOR 
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It took 40 additional trials for the correct primary decision percentage 
to reach 90% of the final percentage. The secondary enemy and friendly 
kill decision percentages took 137 and 56 additional correct primary 
decisions, respectively, to reach 90% of their final percentages. 

With the Commander instructor, the first correct primary decision 
was attained after 221 trials, the first correct secondary enemy kill 
and friendly kill decisions after 194 and 177 correct primary decisions, 
respectively. After 4 7 additional trials, the correct primary decision 
percentage reached 90% of the final. The secondary enemy and friendly 
kill decision percentages reached 90% of the final percentages after 40 
and 111 additional correct primary decisions, respectively. 

With the 4000-plus hour instructor, the first correct primary 
decision was reached after 199 trials, the first correct secondary enemy 
and friendly kill decisions coming after 172 and 161 correct primary 
decisions, respectively. Forty -eight additional trials were needed to 
attain 90% of the final correct primary tactical decision percentage, 32 
and 72 additional correct primary decisions were needed to attain 90% 
of the final correct secondary enemy and friendly decision percentages. 

The 300-plus mission instructor used 206 trials before the first 
correct primary tactical decision was reached, 149 and 177 correct 
primary decisions before the first correct secondary enemy and friendly 
kill decisions, respectively. Eighty-three additional trials were needed 
to attain 90% of the final correct primary tactical decision percentage. 
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Table 16 



BEGINNING STUDENT-AVERAGE INSTRUCTOR 




65 



Figure 17 
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Figure 19 
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Figure 20 
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Figure 22 



BEGINNING STUDENT-4000 HOUR INSTRUCTOR 
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Figure 25 
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104 and 225 additional correct primary decisions to reach 90% of the 
first correct secondary enemy and friendly kill percentages, respectively. 

In comparison with the runs and graphs for the average pilot as 
student, three significant points stand out (See Figures 4-15, 16-27 
and Table VIII): 

(1) The beginner requires a lengthy time to attain even one 
correct decision, no matter v/hat decision is being considered. 

(2) After a correct decision, the beginner in general, needs a 
longer learning time to reach the average pilot's degree of 
sophistication. 

(3) Once this degree of sophistication is reached, the beginner 
and average pilot are nearly identical in behavior. 

In essence, the beginner is, indeed, a beginner. In fact, the 
beginner could even be considered the average pilot three hundred trials 
younger. The panel of aviators considered the beginner as analogous to 
the beginning aviation student: the long, flat, error period being as 
in his ground and basic schooling where the basic ideas and guidance 
are implanted; the learning period as the time when a student’s motor 
skills are honed to the point that unencumbered mental integration of the 
entire flight situation is developed - the transition from flight student 
to a full fledged Naval aviator; and the more constant, steady period 
as the reliable activity of an experienced pilot. 

As can be seen in Table VI, discrepancies do appear between 
instruction as in the average pilot student runs . Whether these dis- 
crepancies arc significant or not is a matter of conjecture. In light 
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of tile data analysis showing little correlation of biographical features 
witli decisions reached, this question of significant instructor differences 
must remain unansv>?ered unless further research is conducted. 

C. THE ARTIFICIAL INTELLIGENCE QUESTION 

Does this program and its resulting runs fall into the classification 
of artificial intelligence? Taking the program in. its entirety, one finds 
that it does analyze its own performances, does diagnose its failures 
and does make changes that enhance its future effectiveness and per- 
formance. This behavior, if witnessed in a human being, would be 
called intelligent behavior. Therefore, by previously cited criteria 
[8, 11] this program does, indeed, fall into the classification of 
artificial intelligence. 
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VI.. CONCLUSION 



In summation , the low correlations betv/een the biographical 
factors of rank, hours and combat missions and the decisions made in 
the evaluation. of an air-to-air combat situation is disconcerting. The 
explanation that the standardization of training and flight procedures 
may account for this lack of relationship seems of doubtful validity 
to the author. The one area showing any hint of a relationship is rank 
and the prediction of friendly kills. Still, the coefficient of correlation 
is weak (See Table II) . 

The multiple correlations between the decisions and situational 
variables are moderately low to moderately high, varying from .3378 to 
.8193. The lack of any high correlation coefficient values may be due 
to randomness in the primary and secondary decision making among the 
pilots. It may also be due to the failure of the multiple regression to 
capture the decision making policy either due to missing variables or 
a failure to capture a non-linear relationship with a linear function. 
Still, in the case of the average pilot, the equations produced from the 
multiple regression predicted 757 correct primary decisions out of 900, 
548 correct enemy kill decisions out of 757, and 714 correct friendly 
kill decisions out of 757. This indicates a rather adequate capturing 
of the average pilot's decision policy. The relatively low correlation 
statistics may, in this light, be misleading. 
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On, the average, the variables most influencing a pilot in reaching 
a tactical decision of the types shown in Figures I and IF appear to be 
MPG and danger, the least important being fuel remaining and rules of 
engagement (See Table VI) . The fact that fuel remaining is considered 
least important while MPG, a function of fuel remaining , is considered 
most important indicates the necessity of considering MPG as a separate, 
though not necessarily independent, function of range and fuel remaining. 

A study by Rigney and De Bow [6] has shown that 63 Combat Infor- 
mation Center watch officers considered range and enemy course (heading) 
as the most important air raid threat variables. Although the watch 
officer's situation is radically different from that of the pilot, the fact 
that range is an argument of the MPG function and course is an argument 
of the danger function is interesting, especially in light of the fact that 
the watch officer does not consider fuel. 

The most important variables considered in making the secondary 
enemy kill decisions are the tactical decision reached, fuel-remaining, 
and rules of engagement (See Table VI). The least important are MPG 
and the number of enemy. An interesting reversal of importance can be 
witnessed in MPG being considered most important in reaching a tactical 
decision and least important in the secondary enemy kill decision. Also, 
the two least important variables in the tactical decision are second and 
third in importance in predicting this secondary enemy kill decision. 
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Number of enemy, tactical decision and speed seem most important 
in predicting pilots' friendly kill expectancies . Least important in this 
prediction are MPG and range. Again, in comparison with the tactical 
decision, MPG is seen to switch roles. Also noticeable is the number 
of enemy's rise to most important while, in the enemy kill decision, it is 
the next to least important. The three decisions appear to be, in fact, 
three separate decisions, the first (primary tactical) affecting the others 
(enemy kills and friendly kills) but neither of the latter two affecting 
each other. 

Looking at the pilots individually, and within their biographical 
groups, differences do appear to exist in how they reach their tactical 
decisions . But these differences do not appear to be correlated with 
rank, hours or missions but are, rather, insignificant random pertur- 
bations. Secondary decisions shov/ little cross-group variation (See 
Table VI and Appendix G) . 

The artificial intelligence program does appear to show adaptive 
learning behavior when the program is taken in its entirety. The behavior 
of the program can reasonably be termed intelligent. Therefore, one 
could safely say that the variables and factors influencing an air-to-air 
combat evaluation can, for the most part, be quantified and utilized in 
an artificial intelligence application of air-to-air combat decision 
making policies . 
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VII. CRITICAL EVAEUATIOLI OF THE STUDY 



Several pertinent critical comments pertaining to this study are: 

(1) The group of 36 pilots from which data were gathered is too small 
to allow any justifiable generalization to Navy pilots in general. 
Consequently, as a result of this small size, any statistics and 
findings generated may not be representative of .Navy pilots in 
general . 

Granted, no generalizations about all Navy fighter pilots can be 
safely made from these findings. An appreciably larger sample 
is required. Still, the sample did cover 2304 separate evaluations. 
Any generalizations drawn do apply, at least, to the group and 
situations studied. 

(2) The assumption that certain factors, such as rank, primary 
decisions, and rules of engagement, can be quantified is ques- 
tionable, i.e. , does a Commander possess five times the amount 
of rank as an Ensign? 

Granted, especially in the area of attempting to quantify a variable 
such as rules of engagement. Still, quantifying rank and placing 
the decisions on a continuum are not necessarily unrealistic. 

Perhaps a better substitute for rank would be lineal number and 
the subjects should be asked to give a number value on a decision 
continuum; the relationships between primary decisions as presented 
not necessarily being linear. 

(3) The situational variables such as bearing, heading, fuel states, 
range and number of enemy are not, in reality, restricted to only 
two states . 

Granted, however, questionnaire size limitations enforces such 
a binary state. Still, a better data base could result from a 
broader representation of the situational variables . 

(4) In light of the low biographical-decision correlation, how can one 
assume that expertise is proportional to a biographical attribute? 

The question of expertise, although interesting, is essentially 
a moot one. The only true criteria of expertise in the situation 
would deal with the results of actual air-to-air combat. Since 
the number of people with this kind of expertise is small and not 
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available to the researcher, "experts” were picked by the previously 
described method.. Whether, as a result of learning, the student 
has developed into a better or worse fighter pilot cannot be deter- 
mined is irrelevant. What is relevant is whether or not the student 
approaches the standards of his declared "expert" instructor. 

(5) It appears unrealistic to ask a pilot with no combat experience to 
evaluate a combat situation. 

Although seemingly unrealistic on the surface, the armed forces 
do require a person with no combat experience to make an evaluation 
of an actual combat situation. However, the person normally has 
undergone extensive and realistic simulated combat training. All 
of the 36 pilots studied have undergone such training. An interes- 
ting note is the similarity in decisions between the no combat 
group and Lieutenant Commander group which has an average 
combat total of over 160 missions (See Table VI and Appendix G) . 

(6) The artificial intelligence program is simply a model of a situation- 
instructor-student interaction. 

Whether the artificial intelligence program is a model or does not 
detract from the fact that it meets the stated criteria of artificial 
intelligence . 

(7) Certain factors may be used for predictors several times in the 
same equation, such as MPG, range and fuel, raising the multiple 
correlation accordingly. 

Granted, but the apparent separation of seemingly related functions 
in the ranking scales (See Table VI and Appendix G) and the exper- 
ience of the panel of aviators does not make this practice appear 
unrealistic . 

(8) The student's confidence level increases with success but does 
not decrease with failure, a situation that may be unrealistic. 

Granted, however, when a decreasing confidence level was imple- 
mented, the modification factor in the beginning stages grew so 
large as to cause erratic and rapid convergence to the instructor's 
coefficients. Overcoming this problem, hovv'ever, would not be 
difficult . 
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Vril.. CONTINUATION OF THE STUDY 



More data are desperately needed before any of the results un- • 
covered can be safely generalized to Navy pilots in general. Likewise, 
more situations are needed for presentation to the subjects. A factor 
indicating a consideration of the enemy's plight must also be sought. 

For example, variables indicative of the friendly airplane pointing at 
the enemy or number of friendlies could be considered. 

The computer graphics field is a likely method to use in future 
data collection and in developing real time interactive artificial intel- 
ligence applications. For example, the cathode ray tube screen could 
easily be configured to resemble a cluster of instruments and radar scope. 
The situation would then be presented on the scope via simulated radar 
images, digital readings on the instrument and appropriate character 
generation. Via random number generation a multitude of situations 
could be presented to the subject. Teletype or light pen input would 
increase the subject's interaction and speed of response, allov/ing 
several hundred responses in the space of an hour. Real time data 
collection and analysis could produce desired statistics and multiple 
regression equations on demand. In the same manner, by using the 
artificial intelligence program described in this study, the human could 
take over the role of instructor with appropriate modifications for response 
collection and analysis. By not utilizing the modification subroutines^ 
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the human could take the place off the student, his coefficients being 
evaluated and graded by the instructor, followed by a response such 
as "You are not emphasizing yonr fuel state enough" or "You are neglecting 
your range consideration." The training, selection and research possi- 
bilities in this area are boundless, limited only by the researcher's 
imagination. Through this kind of work, the fighter pilot may become 
less of an enigma, less of a superman; more of a human. 
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APPENDIX A 





QUESTIONNAIRE 


1. a) 


Relative position (range and bearing) of the 
enemy aircraft to the strike force. 


b) 


Type enemy aircraft suspected or known. 


c) 


r cannot choose between them 


2. a) 


Composition of enemy force — number of raid(s) 
and number of aircraft in each raid . 


b) 


Number of assigned TARCAP aircraft available. 


c) 


I cannot choose between them. 


3 . a) 


Range of enemy aircraft to the strike group. 


b) 


Speed (absolute) of the enemy aircraft. 


c) 


I cannot choose between them. 


4 . a) 


Relative position (range and bearing) of the 
enemy aircraft to the strike group. 


b) 


Range of enemy aircraft to the strike group. 


c) 


I cannot choose between them. 


5. a) 


Type enemy aircraft suspected or known. 


b) 


Capabilities and systems operability of your 
own aircraft (e.g., fuel state, number of 
missiles , etc .) . 


c) 


I cannot choose between them. 


6 . a) 


Speed (absolute) of the enemy aircraft. 


b) 


Altitude of the enemy aircraft. 


c) 


I cannot choose between them. 
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7. a) 


Anti-aircraft, including surface to air mis- 
sile (SAM), activity in your immediatearea . 


b) 


Capabilities and systems operability of your 
own aircraft (e.g. , fuel state, number of 
missiles , etc . ) . 


c) 


I cannot choose between them. 


8 . a) 


Altitude of the enemy aircraft 


b) 


Composition of enemy force — number of raid(s) 
and number of aircraft in each raid. 


c) 


I cannot choose between them. 


9 . a) 


Number of assigned TARCAP aircraft available. 


b) 


Bearing of the enemy aircraft from yourself. 


c) 


I cannot choose between them. 


10 . a) 


Bearing of the enemy aircraft from yourself. 


b) 


Anti-aircraft, including surface to air mis- 
sile (SAM), activity in your immediate area. 


c) 


I cannot choose between them. 


11. a) 


Relative position (range and bearing) of the 
enemy aircraft to the strike force. 


b) 


Rate of closure of enemy aircraft to yourself. 


c) 


I cannot choose between them. 


12 . a) 


Rate of closure of enemy aircraft to yourself. 


b) 


Composition of enemy force — number of raid(s) 
and number of aircraft in each raid . 


c) 


I cannot choose between them. 
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13. a) 


Number of' assigned TARCAP aircraft available 


b) 


Range of enemy aircraft to the strike force. 


c) 


I cannot choose between them. 


14. a) 


Speed (absolute) of the enemy aircraft. 


b) 


Relative position (range and bearing) of the 
enemy aircraft to the strike force. 


c) 


I cannot choose between them. 


15. a) 


Capabilities and systems operability of your 
ovm aircraft (e.g. , fuel state, number of mis- 
siles, etc.). 


b) 


Range of enemy aircraft to the strike force. 


c) 


I cannot choose between them. 


16 . a) 


Speed (absolute) of the enemy aircraft. 


b) 


Type enemy aircraft suspected or known. 


c) 


I cannot choose between them. 


17. a) 


Anti-aircraft, including surface to air mis- 
sile (SAM) , activity in your immediate area . 


b) 


Altitude of the enemy aircraft. 


c) 


I cannot choose between them. 


18. a) 


Capabilities and systems operability of your 
own aircraft (e.g. , fuel state, number of 
missiles , etc .) . 


b) 


Altitude of the enemy aircraft. 


c) 


1 cannot choose between them. 
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19. a) 


Type of enemy aircraft suspected or known. 


b) 


Anti-aircraft, including surface to air mis- 
sile (SAM) , activity in yoiir immediate area . 


c) 


I cannot choose between them. 


20. a) 


Relative position (range and bearing) of the 
enemy aircraft to the strike force. 


b) 


Composition of enemy forces — number of raid(s) 
and number of aircraft in each raid. 


c) 


I cannot choose between them. 


21. a) 


Rate of closure of enemy aircraft to yourself. 


b) 


Range of enemy aircraft to the strike force. 


c) 


1 cannot choose between them. 


22. a) 


Number of assigned TARCAP aircraft available. 


b) 


Relative position (range and bearing) of the 
enemy aircraft to the strike force. 


c) 


I cannot choose between them. 


23. a) 


Capabilities and systems operability of your 
own aircraft (e.g. , fuel state, number of mis- 
siles , etc . ) . 


b) 


Speed (absolute) of the enemy aircraft. 


c) 


I cannot choose between them. 


24. a) 


Capabilities and systems operability of your 
own aircraft (e.g. , fuel state, number of mis- 
siles , etc .) . 


b) 


Number of assigned TARCAP aircraft available. 


c) 


I cannot choose between them. 
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25. a) 


Composition of enemy forces — number of raid(s) 
and number of aircraft in each raid . 


b) 


Bearing of the enemy aircraft from yourself. . 


c) 


I cannot choose betv/een them. 


26. a) 


Type enemy aircraft suspected or known. 


b) 


Bearing of the enemy aircraft from yourself. 


c) 


I cannot choose between them. 


27. a) 


Rate of closure of enemy aircraft to yourself. 


b) 


Anti-aircraft, including surface to air mis- 
sile (SAM), activity in your immediate area. 


c) 


I cannot choose between them. 


28. a) 


Anti-aircraft, including surface to air mis- 
sile (SAM), activity in your immediate area. 


b) 


Range of enemy aircraft to the strike force. 


c) 


I cannot choose between them. 


29. a) 


Altitude of the enemy aircraft. 


b) 


Type of enemy aircraft suspected or known. 


c) 


I cannot choose between them. 


30. a) 


Altitude of the enemy aircraft. 


b) 


Number of assigned TARCAP aircraft available. 


c) 


I cannot choose between them. 
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31. a) 


Bearing of the enemy aircraft from yourself. . 


b) 


Capabilities and system operability of your 
own aircraft (e.g. , fuel state, number of mis- 
siles , etc .) . 


c) 


I cannot choose between them. 


32. (a) 


Composition of enemy force — number of raid (s) 
and number of aircraft in each raid. 


b) 


Type of enemy aircraft suspected or known. 


c) 


I cannot choose between them. 


33. a) 


Bearing of the enemy aircraft from yourself. 


b) 


Rate of closure of enemy aircraft to yourself. 


c) 


I cannot choose between them. 


34. a) 


Composition of enemy force--number of raid(s) 
and number of aircraft in each raid . 


b) 


Anti-aircraft, including surface to air mis- 
siles (SAM), activity in your immediate area. 


c) 


I cannot choose between them. 


35. a) 


Rate of closure of enemy aircraft to yourself. 


b) 


Capabilities and system operability of your 
own aircraft (e.g. , fuel state, number of mis- 
siles , etc .) . 


c) 


I cannot choose between them. 


36. a) 


Number of assigned TARCAP aircraft available. 


b) 


Speed (absolute) of the enemy aircraft. 


c) 


I cannot choose between them. 

0 
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3-7. 



a) 

b) 

c) 

3 8. a) 

b) 

c) 

39 . a) 

b) 

c) 

40 . a) 

b) 

c) 

41. a) 

b) 

c) 

42. a) 

b) 

c) 



Speed (absolute) of the enemy aircraft . 

Anti-aircraft, including surface to air mis- 
siles (SAM), activity in your immediate area. 

I cannot choose between them. 

Range of enemy aircraft to the strike force. 

Altitude of the enemy aircraft. 

I cannot choose between them. 

Type enemy aircraft suspected or known. 

Number of assigned TARCAP aircraft available. 

I cannot choose between them. 

Altitude of the enemy aircraft. 

Bearing of the enemy aircraft from yourself. 

I cannot choose between them. 

Relative position (range and bearing) of the 
enemy aircraft to the strike force. 

Capabilities and systems operability of your 
own aircraft (e.g. , fuel state, number of mis- 
siles , etc .) . 

I cannot choose between them. 

Speed (absolute) of the enemy aircraft. 

Rate of closure of enemy aircraft to yourself. 

I cannot choose between them. 
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43 .. 



43.. a) 


Anti-aircraft' , including surface to air mis- 
sile (SAM) , .activity in your immediate area . 


b) 


Number of assigned TARCAP aircraft available . 


c.) 


I. cannot choose between them. 


44.. a) 


Anti-aircraft, including surface to air mis- 
sile (SAM), activity in your immediate area. 


b) 


Relative position (range and bearing) of the 
enemy aircraft to the strike force. 


c) 


r cannot choose between them. 


45. a) 


Range of enemy aircraft to the strike force. 


b) 


Bearing of the enemy aircraft from yourself. 


c) 


I cannot choose between them. 


46. a) 


Rate of closure of enemy aircraft to yourself. 


b) 


Altitude of the enemy aircraft. 


c) 


I cannot choose between them. 


47. a) 


Capabilities and systems operability of your 
own aircraft (e.g. , fuel state, number of mis- 
siles, etc.). 


b) 


Composition of enemy force — number of raid(s) 
and number of aircraft in each raid. 


c) 


I cannot choose between them. 


48. a) 


Range of enemy aircraft to the strike force . 


b) 


Composition of enemy force — number of raid(s) 
and number of aircraft in each raid. 


c) 


I cannot choose betv/een them. 
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49 . 



49. a) 


Bearing of the enemy aircraft from yourself. 


b) 


Relative position (range and bearing) of the 
enemy aircraft to the strike force. 


c) 


I cannot choose between them. 


50. a) 


Bearing of the enemy aircraft from yourself. 


b) 


Speed (absolute) of the enemy aircraft. 


c) 


I cannot choose between them. 


51. a) 


Range of enemy aircraft to the strike force. 


b) 


Type enemy aircraft suspected or known. 


c) 


I cannot choose between them. 


52. a) 


Type enemy aircraft suspected or known. 


b) 


Rate of closure of enemy aircraft to yourself. 


c) 


I cannot choose between them. 


53. a) 


Altitude of the enemy aircraft. 


b) 


Relative position (range and bearing) of the 
enemy aircraft to the strike force. 


c) 


1 cannot choose between them. 


54. a) 


Composition of enemy force — number of raid(s) 
and number of aircraft in each raid. 


b) 


Speed (absolute) of the enemy aircraft. 


c) 


I cannot choose between them. 


55. a) 


Number of assigned TARCAP aircraft available. 


b) 


Rate of closure of enemy aircraft to yourself. 


c) 


I cannot choose between them. 
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APPENDIX B 





PILOTS RESPONDING TO MAIL 


-OUT OUESTIONNAIRE 




Rank 


Hours 


Missions 


1. 


Commander 


4500 


70 


2. 


Commander 


4000 


143 


3 . 


Commander 


3500 


240 


4. 


Commander 


3300 


354 


5. 


Lieutenant Commander 


3450 


200 


6. 


Lieutenant Commander 


3200 


300 


7. 


Lieutenant Commander 


3200 


140 


8. 


Lieutenant Commander 


3000 


218 


9. 


Lieutenant Commander 


2700 


172 


10. 


Lieutenant Commander 


2655 


130 


11. 


Lieutenant Commander 


2550 


113 


12. 


Lieutenant Commander 


2500 


11 


13. 


Lieutenant Commander 


1900 


150 


14. 


Lieutenant Commander 


1600 


200 


15. 


Lieutenant 


2350 


80 


16. 


Lieutenant 


1800 


249 


17. 


Lieutenant 


1.800 


220 


18. 


Lieutenant 


1750 


70 


19. 


Lieutenant 


1700 


154 


20. 


Lieutenant 


1700 


75 


21. 


Lieutenant 


1700 


40 


22 . 


Lieutenant 


1500 


200 


23. 


Lieutenant 


1500 


100 


24 . 


Lieutenant 


1400 


150 


25 . 


Lieutenant 


1400 


55 


26. 


Lieutenant 


1300 


90 


27. 


Lieutenant 


1150 


100 


28. 


Lieutena nt 


1100 


130 


29. 


Lieutenant 


1100 


129 


o 

CO 


Lieutenant 


1050 


0 


31 . 


Lieutenant 


1000 


150 


32 . 


Lieutenant 


850 


0 


33. 


Lieutenant 


750 


0 


34. 


Lieutenant 


750- 


0 


35. 


Lieutenant 


730 


0 


36. 


Lieutenant (junior grade) 


450 


0 
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APPENDIX C 



PANEL OF AVIATORS 



Commander (selectee) - pilot, attack and fighter type aircraft experience; 
two instructor tours in the basic training command; one 
instructor tour with an attack Replacement Air Group; three 
combat tours in Southeast Asia; candidate for M.S. in 
Computer Science. 

Lieutenant Commander - pilot; patrol type aircraft experience; one 

instructor tour in the basic training command; one combat 
tour in Southeast Asia; candidate for M.S. in Computer 
Science . 

Lieutenant - pilot; attack type aircraft experience; one instructor tour 
in the Replacement Air Group; one combat tour in Southeast 
Asia; candidate for M.S. in Computer Science. 

Lieutenant - Naval Flight Cfficer; experience as airborne intercept 

controller; one instructor tour in the basic training command 
teaching computer fundamentals; one combat tour in South- 
east Asia; candidate for M.S. in Computer Science. 

Lieutenant - Naval Flight Cfficer; experience as airborne intercept 

controller; one tour working on airborne computer display 
systems; one combat tour in Southeast Asia; candidate for 
M.S. in Computer Science. 
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APPENDIX D 



DISCRIMINANT ANALYSIS FOR GROUPS DIFFERENTIATED 
BY biographical DATA 



Significance Level of .05 

Grouped by Rank: 

Chi square = 14.0 
Degrees of Freedom = 21 

No significant differences between groups differentiated 
by rank . 

Grouped by Hours: 

Chi square = 24.0 
Degrees of Freedom = 28 

No significant difference between groups differentiated by 
flight hours . 

Grouped by Missions: 

Chi square = 44.0 
Degrees of Freedom = 28 

No significant difference between groups differentiated 
by combat missions. 
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APPENDIX E 



SELECTED PREDICTOR EQUATION COEFFICIENTS 
Primary Tactical Decision — Corresponds to Prico Vector: 





Average Pilot 


CDR 


4000+ 


300+ 


I.. 


1 ..51835 


1.23987 


1.74982 


1 . 12484 


2 


-.,00014 


-.00005 


-.00015 


-.00005 


3-.. 


.3910 


.05223 


.03590 


.05204 


4',.. 


..05065 


.03086 


.05952 


.03043 


5 .. 


-..12239 


-. 07430 


-■.08141 


-.06924 


6.. 


.,00352 


.00223 


.00132 


.00440 


7., 


.00028 


.00023 


.00016 


.00079 


8.. 


8.24161 


5.59970 


4.11829 


-3.22020 


Secondary Enemy Kill Decision - 


Corresponds to Secoen Vector: 




Average Pilot 


CDR 


4000+ 


300+ 


1. 


.82003 


1.20750 


.23976 


1.06406 


2 . 


.00034 


.00018 


.00038 


.00024 


3 . 


.02213 


.05697 


.05474 


.01154 


4.. 


.03087 


. 12087 


-.01666 


.18579 


5. 


.41223 


.20282 


.34197 


.21952 


6. 


-.00281 


-.00306 


-.00242 


-.00253 


7. 


-.00020 


-.00001 


.00013 


-.00008 


8. 


-7.81100 


-13 .56913 


27.20380 


11.08119 


9. 


-.43846 


-.66584 


-.22533 


-.57411 


Secondary Friendly Kill Decision 


- Corresponds 


to Secofr Vector: 




Average Pilot 


CDR 


4000+ 


300+ 


1 . 


.56915 


.39760 


.74056 


.17621 


2 . 


-.00004 


.00012 


.00009 


.00007 


3 . 


-.00558 


-.01702 


-.01714 


-.01947 


4 . 


.08816 


.18036 


.08989 


.30649 


5. 


-.09740 


-.23837 


-.25172 


-.26590 


6. 


.00059 


.00129 


-.00023 


.00278 


7. 


.00013 


.00019 


.00014 


.00024 


8. 


3 .26289 


28.96692 


25.10820 


24.35280 


9. 


-.24901 


-.32516 


-.33688 


-.28021 
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APPENDIX F 



MEAN AND STANDARD DEVIATIONS OF DECISION S MADE 



Rank Group 


Primary Decision 


Enemy Kills 


Friendly Kills 


CDR 


X 


r.S5 


L.40 


.61 




SD 


.8,2 


M09 


1.01 


LCDR 


X 


1.36 


.93 


.32 




SD 


.,80 


.99 


.64 


LT 


X 


2.01 


i: 10 


. 18 




SD 


.78 


MOO 


.48 


LTJG 


X 


2 .16 


.70 


.08 




SD 


..76 


.75 


.27 


Hours Group 








4000+ 


X 


2.16 


.83 


.41 




SD 


.80 


.78 


.55 


4000 


X 


1.83 


1.44 


.51 




SD 


.79 


1.15 


.95 


3000 


X 


1.92 


.96 


.26 




SD 


.79 


.94 


.61 


2000 


X 


2.07 


1.04 


.16 




SD 


.96 


1.06 


.45 


1000 


X 


1.99 


1.00 


.26 




SD 


.77 


.84 


.55 


Mission 


Group 








300+ 


X 


1.74 


1.56 


.88 




SD 


.92 


1.02 


1.28 


300 


X 


2.02 


1.10 


.19 




SD 


.74 


1.08 


.53 


200 


X 


2.01 


.96 


.23 




SD 


.80 


1.01 


.53 


100 


X 


2.01 


1.10 


.23 




SD 


.80 


1.04 


.57 


0 


X 


1.97 


1.07 


.24 




SD 


.75 


.85 


.52 



X = arithmetic mean 
SD = standard deviation 
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APPENDIX G 



VARIABLE RANKINGS FOK HOUR AND MISSION GROUPS 



HOUR GROUP: 

Primary Tactical Decision 





4000 + 


3000-4000 


2000-3000 


1000-2000 


0-1000 


MPG 


1 


5 


7 


1 


7 


Danger 


5 


2. 


2 


3 


3 


Speed 


3 


3 


3 


4 


2 


Range 


4 


U 


1 


6 


1 


Number of Enemy 


2 


4 


4 


5 


5 


Rules of 
Engagement 


7 


6 


6 


7 


6 


Fuel Remaining 


6 


T 


5 


2 . 


4 



Secondary Enemy Kill Decision 



IDECIS 


4000+ 

3 


3000-4000 

1 


2000-3000 

1 


1000-2000 

2 


0-1000 

1 


Fuel Remaining 


1 


4 


4 


1 


2 


Rules of 
Engagement 


4 


5 


2 


3 


4 


Danger 


5 


3 


3 


4 


5 


Speed 


6 


6 


5 


5 


3 


Range 


2 


7 


6 


6 


- 


Number of Enemy 


7 


2 


7 


7 


7 


MPG 


8 


8 


- 


8 


6 


Secondary Friendly 
4000+ 

Number of Enemy 2 


Kill Decision 

3000-4000 

1 


2000-3000 

2 


1000-2000 

1 


0-1000 

1 


IDECIS 


1 


2 


1 


2 


2 


Speed 


4 


4 


3 


3 


3 


Rules of 
Engagement 


3 


5 


6 


7 


5 


Fuel Remaining 


7 


6 


5 


8 


4 


Danger 


8 


3 


7 


6 


6 


Range 


6 


7 


4 


4 


7 


MPG 


6 


8 


- 


5 


8 
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APPENDIX G (continued) 



VARIABLE RANKINGS FOR HOUR AND MISSION GROUPS 
MISSION GROUP: 



Primary Tactical Decision 



3.00+ 



MPG 7 

Danger 3 

Speed I 

Range 2 

Number of Enemy 4 

Rules of 

Engagement 5 

Fuel Remaining 6 



200-300 

i; 

2 : 

3 

4 
5 ^ 

r 

6 



Secondary Enemy Kill Decision 



300+ 



IDECIS 1 

Fuel Remaining 4 

Rules of 

Engagement 6 

Danger 5 

Speed 7 

Range 3 

Number of Enemy 2 

MPG 8 



200-300 

1 

2 . 

4 
3- 

7 
6 

5 

8 



Secondary Friendly Kill Decision 



300+ 



Number of Enemy 1 

IDECIS 5 

Speed 6 

Rules of 

Engagement 3 

Fuel Remaining 8 

Danger 4 

Range 2 

MPG 7 



200-300 

2 

1 

7 

6 

4 
3 

8 

5 



100-200 

1 

2 . 

3 

s: 

4 

6 

7 



100-200 

1 

2 

3 

4 
6 

5 
7 



100-200 

1 

2 

3 

6 

8 

7 

4 

5 



1-100 

1 

2 

5 
4 
3 

6 
7 



1-100 

1 

2 

3 

4 

5 

6 

7 

8 



1-100 

2 

1 

3 

4 

5 

6 
8 
7 



APPENDIX H 



MULTIPI.E CORRELATIONS OF PREDICTOR EQUATIONS 
WITH PREDICTED DECISION 





Primary Tactical 


Secondary Enemy 


Secondary Friendly 




Decision 


Kill Decision 


Kill Decision 


Average Pilot 


.6032 


.6027 


.4015 


CDR 


.5794 


.5859 


.4391 


LCDR 


.6236 


.5654 


.5299 


LT 


.6101 


.6527 


.3732 


LT(jg) 


.7568 


.8193 


.5284 



Hours 
4000 + 


.4587 


. 7930 


.5840 


3000-4000 


.6775 


.5638 


.4303 


2000-3000 


.6249 


.5988 


. 5217 


1000-2000 


.6816 


.6755 


.3378 


0-1000 


.6062 


.6190 


.4442 



Missions 

300+ 


.7459 


.7052 


.5128 


200-300 


.6402 


.5748 


.4524 


100-200 


.5778 


.5915 


.3397 


1-100 


.6309 


.6677 


.5309 


0 


.6213 


.6143 


.4499 
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DIMENSION PT(900), ET ( 900 ) , F T ( 900 ) , TK ( 900 ) 



COMPUTER PROGRAM 
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IS THE ENEMY POINTING AT THE FRIENDLY? 
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102 IF( STARSF( J) .GE.SECOFRt J) ) GO TO 104 
KORF { J )=1 
GO TO 1001 

104 IF(STARSFIJ) ,EQ.SECOFR(J) ) GO TO 106 
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